National Assessment Of Educational Progress Research Articles

Text-to-Speech Technology and Math Performance: A Comparative Study of Students With Disabilities, English Language Learners, and Their General Education Peers

This study investigates the relationship between text-to-speech (TTS) usage and item-by-item performance in the 2017 eighth-grade National Assessment of Educational Progress (NAEP) math assessment, focusing on students with disabilities (SWDs), English language learners (ELLs), and their general education (GE) peers. Results indicate that all students use TTS more for longer and more difficult math items as well as for multiple-choice or short-response formats. Among SWDs and GE students, lower math proficiency and higher perceived time pressure are linked to higher TTS usage. Moreover, among GE students, factors such as male gender, minority status, lower math persistence, and higher math interest and effort during testing contribute to higher TTS usage. TTS usage is positively associated with item performance for SWDs and ELLs who received extended time accommodations but not for those who did not receive such accommodations or for general education students. The study suggests that the time constraints of speeded digital assessments may limit the potential benefits of TTS for SWDs and ELLs in math problem-solving.

Closing Reporting Gaps: A Comparison of Methods for Estimating Unreported Subgroup Achievement on NAEP

ABSTRACT The National Assessment of Educational Progress (NAEP) program only reports state-level subgroup results if it samples at least 62 students identifying with the subgroup. Since some subgroups constitute small proportions of many states’ general student populations, these minority subgroups are seldom sufficiently sampled to meet this sample size requirement. Consequently, education researchers and policymakers are regularly left without a comprehensive understanding of how states are supporting the learning and achievement of different subgroups of students, including underserved subgroups. Using grade 8 mathematics results in 2015, this study compares the performance of two separate techniques in predicting mean subgroup achievement on NAEP. Results demonstrate that a small area estimation technique, known as the Fay-Herriot (FH) model, represents a promising approach. Follow-up research should employ the FH technique with math achievement data from other testing years, as well as data from NAEP reading to evaluate the technique’s generalizability.

Reducing Inequality in Student Outcomes in U.S. Geography Education: The Importance of Understanding Student Attitudes

This study examines relationships between geography achievement and the psychological characteristics of geography learners in eighth grade. Using data provided by the National Assessment of Educational Progress (NAEP), the researchers developed a statistical model of geography achievement comprising student and teacher/school-level predictor variables. The model included five predictor variables based on composite measures of student affective disposition. Hierarchical linear modeling accounted for data clustering. Controlling for the full set of predictors, geography achievement was positively associated with students’ self-reported enjoyment of complex problems, academic self-discipline, and interest in and enjoyment of geography, whereas lower levels of geography achievement were associated with students’ self-reported level of persistence in learning. Race, gender, disability, minority language status, and instructional exposure were predictive of affective differences between groups of geography learners. The results of the study underscore the need to consider student attitudes in efforts to close achievement gaps and encourage the future geography intentions of students in U.S. geography education.

The Intersection of Language and Disability Progress of English Learners With Disabilities on NAEP Reading

Despite evidence of unequal learning opportunities, there is limited understanding of how English learners (ELs) with disabilities are faring academically nationwide in comparison with their EL and student with disability counterparts. In response, we conducted secondary data analyses of the 2009 to 2019 National Assessment of Educational Progress (NAEP) reading assessments. Compared to their peers, ELs with disabilities performed lower in both Grades 4 and 8. There is a visible improvement in this important student population’s performance on the NAEP 2015 reading assessment—a year in which disaggregated reporting for ELs with disabilities performance became mandatory. These findings have implications for improved instructional, assessment, and program supports to serve ELs with disabilities.

Extended Time Accommodation and the Academic, Behavioral, and Psychological Outcomes of Students With Learning Disabilities.

This study analyzed performance, process, and survey data of eighth graders with learning disabilities (LDs) who took the 2017 National Assessment of Educational Progress (NAEP) digital math test. Compared with students with LDs who did not receive extended time accommodations (ETAs), students with LDs who received and used ETA scored significantly higher on the test, whereas students with LDs who received but did not use ETA scored significantly lower on the test. In addition, students with LDs in the two ETA groups reported a lower level of perceived time pressure and a higher level of math interest and enjoyment than their peers who did not receive ETA. For students with LDs who received ETA, optimal performance was achieved with 50% additional time, while their peers who did not receive ETA typically performed best when utilizing most of their allotted time. The analysis of process data revealed that students with LDs who used ETA performed more actions, had a higher number of revisits, used universal design digital tools more frequently, and performed better on time-consuming items than their peers who did not receive ETA at the same level of math performance.

Driving Assessment Scores with Effective Calculator Implementation

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. The paper examined the impact of calculator exposures on eighth-grade students' 2019 NAEP mathematics assessment scores. To better understand the impact of calculator use on the mathematics achievement of eighth-grade students, this study used a quantitative descriptive research design to analyze secondary data extracted from the 2019 NAEP data set. The findings are: (1) the frequency of calculator use does not impact assessment scores during math lessons. (2) Students who practice using calculators on math tests and quizzes have higher math assessment scores. (3) Frequent primary calculator use can impact math assessment scores. (4) Students who use graphing calculators frequently score higher on math assessments. In conclusion, these findings indicate that using calculators in mathematics is impactful; however, teachers must be trained to implement them effectively.

The footprint of the COVID-19 pandemic in reading performance of students in the U.S. with and without disabilities

BackgroundMany schools around the world involuntarily began remote learning in March of 2020 as a result of the COVID-19 pandemic, which brought immediate changes and challenges to teaching and learning that are likely to influence student achievement into the foreseeable future. While large numbers of students face difficulties in learning to read under typical conditions, remote learning resulted in substantial deficits in reading outcomes. AimsThis study aimed to examine the relative impact of the pandemic on reading performance in United States (U.S.) for students with and without disabilities. Methodsand procedure: In this interpretive analysis, we compared the progress of a large sample of students (n = 219,500) by using the National Assessment of Educational Progress (NAEP) data, in two grade levels (Grades 4 and 8), with and without disabilities, at two time points (2019 and 2022). Outcomes and resultsStatistically significant decreases were identified on the reading outcomes of students in U.S. Students without disabilities experienced a significant decrease in their reading outcomes pre- and post-pandemic, compared to students with disabilities (SWDs). Conclusions and implicationsWe discuss the effects of the COVID-19 pandemic on students’ reading performance and the implications for effective reading instructions in the future.

Socioeconomic Status & Health Disparities: Utilizing a Composite Index across Health Datasets

Research indicates that the relationship between socioeconomic status (SES) and health outcomes are robust, though no ‘gold standard’ as to what best captures SES exists. Many studies use individual proxies to control for SES, but there are a number of limitations to doing so. Additionally, little research has been conducted as to how to develop a US-based composite SES index that can be utilized across various datasets. The aim of the study is to generate a weighted SES index, following the National Assessment of Educational Progress (NAEP)’s call for an understanding of SES groupings and the guidance of the Bureau of Justice’s SES index. After generating a composite SES index, we evaluate whether or not each unique index delivers a similar classification of SES across four national datasets and empirically compare the resulting grouping across datasets utilizing multiple regression analyses. Using weighted descriptive statistics and weighted logistic regression analyses, we reveal the key variations in the distribution of determinants of the socioeconomic index. While previous studies conflated race and ethnicity with SES to resulting health indicators, this study captures both, highlighting the importance of each covariate and a composite index. Our findings suggest similar outcomes with regards to the assigned SES classifications (i.e. low, middle, and high SES) across the four national datasets.

COVID Learning Loss: A Call to Action

The COVID-19 pandemic and policy responses designed to mitigate transmission have caused deep and persistent mathematics learning loss among K–12 students. While initial data might have been read optimistically as a blip that would reverse once schools returned to normal, 2023 data from the National Assessment of Educational Progress (NAEP) show that losses persist. While the NAEP does not directly measure quantitative reasoning (QR), the data present a disturbing picture for QR instruction and call for new lines of research that inform QR pedagogical response.

Education Freedom and Student Achievement: Is More School Choice Associated with Higher State-Level Performance on the NAEP?

ABSTRACT School choice is spreading across the U.S. at the same time that scores on the National Assessment of Education Progress (NAEP) are stagnant. We examine the association between education freedom and 8th grade student NAEP score levels and gains in states in 2019. We construct the 2019 Education Freedom Index as a composite measure of the accessibility of private school choice, public charter schools, homeschooling, and public school choice in each state. Using statistical regression, we find that more education freedom is significantly associated with increased NAEP scores and gains, supporting the claim that choice and competition improves system-wide achievement.

Examining the Impact of a Cognitive Strategies Approach on the Argument Writing of Mainstreamed English Learners in Secondary School

The stagnation of National Assessment of Educational Progress (NAEP) Writing scores demonstrates the need for research-based instruction that improves writing for all students, especially English learners. In this article, we synthesize the literature on effective instructional practices for this diverse group of learners and describe how these strategies are leveraged in a teacher professional development program that has been previously shown to improve students’ argument writing. Then, we share results of a study that focuses on distinct subgroups of secondary English learners students to (a) determine their needs and challenges and (b) examine the impact of a cognitive strategies approach on rhetorical and linguistic aspects of writing at posttest. Results show English learners have considerable challenges with higher-order tasks involved in writing literary arguments and with the linguistic demands of academic writing before receiving the intervention. However, after receiving the intervention, using descriptive statistics and multiple hierarchical linear regression, we show that these students grew in the areas of presentation of ideas, organization, evidence use, and language use. For example, students designated as reclassified English learners (RFEP [Reclassified Fluent English Proficient]) and students who have even more limited English proficiency (designated as EL [English learner] here) show improvements in many aspects of writing, especially in their ability to write claims and use evidence. In contrast, improvements in language use components were more limited for both groups of learners. Moreover, some of the gains due to being in the treatment were significant enough to bring the average EL student close to parity or beyond their EO (English Only) / IFEP (Initial Fluent English Proficient) peers in the control condition at posttest. We conclude by discussing pedagogical implications for English learners.

THE IMPACT OF EIGHTH GRADERS’ SELF-REPORTED METACOGNITIVE READING STRATEGIES ON 2019 NAEP INFORMATIONAL READING SCORES

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) dataset. It examined the impact of eighth graders’ self-reported metacognitive reading strategies on their 2019 NAEP informational reading scores. A quantitative descriptive research design was utilized to analyze secondary data extracted from the 2019 NAEP dataset. The findings are: (1) The average subscale score of students who feel that they definitely can recognize when they don’t understand something they are reading is significantly higher than those who reported a lower ability level. (2) The average subscale score of students who feel that they definitely can figure out the meaning of a word they don’t know by using other words in the text is significantly higher than those who reported a lower ability level. (3) The average subscale score of students who feel that they definitely can identify the main idea of a text is significantly higher than those who reported a lower ability level. These findings indicate that metacognitive reading strategy instruction could be beneficial to students’ informational reading comprehension, and therefore, the need for further educator professional development on metacognitive reading instruction is warranted.

The Effects of Rating Designs on Rater Classification Accuracy and Rater Measurement Precision in Large-Scale Mixed-Format Assessments.

In standalone performance assessments, researchers have explored the influence of different rating designs on the sensitivity of latent trait model indicators to different rater effects as well as the impacts of different rating designs on student achievement estimates. However, the literature provides little guidance on the degree to which different rating designs might affect rater classification accuracy (severe/lenient) and rater measurement precision in both standalone performance assessments and mixed-format assessments. Using results from an analysis of National Assessment of Educational Progress (NAEP) data, we conducted simulation studies to systematically explore the impacts of different rating designs on rater measurement precision and rater classification accuracy (severe/lenient) in mixed-format assessments. The results suggest that the complete rating design produced the highest rater classification accuracy and greatest rater measurement precision, followed by the multiple-choice (MC) + spiral link design and the MC link design. Considering that complete rating designs are not practical in most testing situations, the MC + spiral link design may be a useful choice because it balances cost and performance. We consider the implications of our findings for research and practice.

EXPLORING THE RELATIONSHIP BETWEEN AFFECTIVE DISPOSITION AND NAEP MUSIC SCORES

This study presented a secondary analysis of the National Assessment of Educational Progress (NAEP) data set. The paper explored the relationship between affective disposition and average scores on the 2016 NAEP music assessment for eighth-grade students. To gain a better understanding of the relationship between students' affective disposition and average music scores, this study used a quantitative descriptive research design to analyze the secondary data extracted from the 2016 NAEP data set. The findings include (1) students who agreed that they had a talent for music had significantly higher average music scores. (2) Students who had been told by others that they are good musicians had significantly higher average music scores. (3) Students who reported that they liked playing music for others had significantly higher average music scores. These findings indicate that budget constraints and cuts to music education in public schools may limit student intelligence and the development of a positive self-concept.

Using sequence mining to study students’ calculator use, problem solving, and mathematics achievement in the National Assessment of Educational Progress (NAEP)

Using appropriate tools strategically to aid in problem solving is a crucial skill identified in K-12 mathematics curriculum standards. As more assessments transition from paper-and-pencil to digital formats, a variety of interactive tools have been made available to test takers in digital testing platforms. Using onscreen calculators as an example, this study illustrates how process data obtained from student interactions with a digitally-based large-scale assessment can be leveraged to explore how and how well test takers use interactive tools and unveil their mathematical problem-solving processes and strategies. Specifically, sequence mining techniques using the longest common subsequence were applied on process data collected from a nationally representative sample who took the National Assessment of Educational Progress (NAEP) mathematics assessment to examine patterns of eighth-grade students’ calculator-use behaviors and the content of calculator input across a series of items. Sequences of keystrokes executed on the onscreen calculator by test takers were compared to reference sequences identified by content experts as proficient and efficient use to infer how well and how consistently the calculator was used. Results indicated that calculator-use behaviors and content differed by item characteristics. Students were more likely to use calculators on calculation-demanding items that involve intensive and complex computations than on items that involve simple or no computation. Using the calculator on more calculation-demanding items and using it in a manner that is more efficient and more similar to reference sequences on these items were related to higher mathematical proficiency. Findings have implications for assessment design and can be used in educational practices to provide educators with actionable process-related information on tool use and problem solving.

Advances and Missed Opportunities in the Development of the 2026 NAEP Reading Framework

The National Assessment of Educational Progress (NAEP) in Reading aims to measure reading comprehension for students in the United States and to monitor progress in our education system. NAEP Reading is developed based on an assessment framework document that is periodically revised to reflect the latest understandings about reading comprehension and its assessment. A key goal of the Visioning Panel (VP) and the Development Panel (DP) charged with updating the NAEP Reading Assessment Framework for 2026 was to lay the foundation for an assessment that made progress toward greater fairness, equity, and validity. In this essay, we discuss how the Framework development process unfolded and its results. We document the unusual ways the National Assessment Governing Board (“the Board”) shaped the development process. We provide evidence that a small group of Board members aimed to preserve the status quo in reading assessment by downplaying reliance on expertise and authoritative sources of research on reading, learning, and assessment and by removing attention to equity in NAEP Reading. We also discuss both successful (i.e., approved by the Board) and unsuccessful (i.e., rejected by the Board) recommendations for changes to the 2026 Framework that initially were proposed by the DP. We end by considering how, despite the efforts of the small group of Board members, we as a literacy field can improve the nature and impact of our large-scale reading assessments, and NAEP in particular.

Extended Time Accommodations on the 2017 NAEP Grade 8 Mathematics Test: Eligibility, Use, and Benefit

Extended testing time is a common accommodation given to students with disabilities. However, little is known about whether students who are eligible for the accommodation actually use it, and with what effect. The present study used process data available from the digitally delivered 2017 National Assessment of Educational Progress (NAEP) mathematics test to explore these topics. We found that although the majority of students with special education plans were eligible for extended time accommodations, most of those eligible students did not actually take any additional time at all to complete the test. We describe differences in eligibility for extended time and use of extended time that were identified according to gender, race/ethnicity, disability type, family income level, English Learner status, perceived importance of the test, and use of the text-to-speech (TTS) accessibility feature. Use of extended time (among eligible students) was associated with higher performance. Implications for practice and future research are discussed.

School vouchers for greater educational equity

Providing public funding to parents to pay children's tuition at private and religious schools has been a matter of debate over the last three decades based on claims such as the lack of accountability in private schools, an increase in segregation, and the fact that it contradicts the principle of the separation between church and state. Such concerns are legitimate, but they need to be discussed considering the new literature that proves the efficacy of private schools, both religious and non-religious, in terms of student achievement, test results, and graduation rates. A thorough look at the test results revealed by the National Assessment of Educational Progress (NAEP) over the last decade indicates that students' scores in private schools are higher than their peers in public schools. NAEP results and the recent empirical studies conducted in states like Florida, Wisconsin, and Washington DC confirmed that the voucher system works well. This information provides compelling evidence that policymakers need to review the private school voucher policies. Hence, this article highlights the efficacy of school vouchers based on recent empirical studies and the academic results of the students enrolled in school?voucher programs. The available data assures that there is no point in depriving students of their right to choose private or religious schools as their peers who choose charter and magnet schools.

Geography Achievement and Future Geographers

Broadening participation in scientific fields, conceptually, is about enhancing the involvement and academic outcomes of underrepresented groups at all levels of education. In recent decades the American Association of Geographers (AAG) and geography departments have collaboratively worked to increase the representation of women, minorities, and persons with disabilities among undergraduate students, graduate students, and the academic and nonacademic professional workforce. Efforts to broaden participation in geography are ongoing, yet mostly continue to exclude K–12 educators and students. Using multiple data sets provided by the National Assessment of Educational Progress (NAEP), this article examines the extent to which students in secondary education are acquiring knowledge representing the distinctive substance and thought processes of geography. NAEP item maps for different student groups were created to analyze varying levels of competency over geography subject matter. Results indicate most students do not know enough geography to think geographically at a high level of proficiency, with persisting achievement gaps associated with race, gender, and other student characteristics. Social disparities in geography achievement are symptomatic of widespread and systemic barriers to geography’s powerful epistemic knowledge, underscoring the need for geographers to adopt a comprehensive research-based strategy for broadening participation that bridges K–12 and postsecondary education.

Student Engagement on the National Assessment of Educational Progress (NAEP): A Systematic Review and Meta-Analysis of Extant Research

ABSTRACT This systematic review examines empirical research about students’ motivation for NAEP in grades 4, 8, and 12 using multiple motivation constructs, including effort, value, and expectancy. Analyses yielded several findings. First, there are stark differences in the perceived importance of doing well on NAEP among students in grades 4 (86%), 8 (59%), and 12 (35%). Second, meta-analyses of descriptive data on the percentage of students who agreed with various expectancy statements (e.g., “I am good at mathematics”) revealed minimal variations across grade level. However, similar meta-analyses of data on the percentage of students who agreed with various value statements (e.g., “I like mathematics”) exposed notable variation across grade levels. Third, domain-specific motivation has a positive, statistically significant relationship with NAEP achievement. Finally, some interventions – particularly financial incentives – may have a modest, positive effect on NAEP achievement.